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Abstract 

Low retention rates in online freshman-level mathematics courses are a concern, especially at 
postsecondary institutions that serve academically unprepared students. The purpose of this study was to 
determine if student performance in an online College Algebra course that relies heavily on text-based 
multimedia tools can be improved by replacing publisher-generated educational resources with instructor¬ 
generated video lectures. The original online College Algebra course placed the publisher-generated 
educational resources in the role of content provider by enabling all publisher-generated learning aids 
within the online homework system and treating instructor-generated educational materials as 
supplemental resources. In contrast, the redesigned online College Algebra course enhanced the course 
instructor’s teaching presence by requiring students to complete instructor-generated guided note-taking 
sheets while watching instructor-generated video lectures, treating publisher-generated learning aids as 
supplemental resources by removing them from within the online homework system. Results indicate 
students who enrolled in a redesigned online College Algebra course that strategically placed the 
instructor in the role of content provider performed significantly better on both online and handwritten 
assessments than did students who enrolled in an online College Algebra course that placed the publisher¬ 
generated educational resources in that role. 


Introduction 

The performance-funding approach to financing higher education has concerned college and 
university administrators nationwide. State appropriations depend upon the institution’s ability to meet or 
exceed performance indicators such as student retention and graduation rates. Given the evidence 
suggesting that withdrawal from college is associated with students’ lack of success in “gatekeeper” 
courses such as entry-level college mathematics courses (Chen, 2013), it is no surprise faculty members 
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who teach freshman-level mathematics are the first to be visited by administrators seeking lower failure 
and withdrawal rates. And, although 46% of chief academic officers at public institutions of higher 
education believe retaining students in the online environment is more difficult than retaining students in 
face-to-face formats (Allen & Seaman, 2014), administrators continue to ask faculty to develop more 
online entry-level college mathematics courses. 

As mathematics educators become more comfortable with emerging technologies that offer 
enhanced teaching and learning opportunities, they are more receptive to administrative requests for 
additional online general studies mathematics courses such as College Algebra and Elementary Statistics. 
Indeed, fully online general studies mathematics courses have become more widespread, in part due to the 
development of text-based, interactive multimedia tools such as ALEKS, MyMathLab, and Enhanced 
WebAssign. As noted on their product websites, these text-based tools have been adopted by all major 
academic publishers and integrated with hundreds of textbooks, serving thousands of educational 
institutions and millions of students (ALEKS, 2014; MyLab and Mastering, 2014; WebAssign, 2014). 
Students enrolled in mathematics courses utilizing these text-based tools have access to an e-book, an 
online homework system, and a variety of embedded publisher-generated learning aids (e.g., video 
lectures, animations, completed examples, and guided tutorials). Mathematics faculty who adopt these 
text-based tools for their face-to-face and online courses have access to an online course management 
system that provides them with a complete set of relevant mathematics exercises from which to create 
online assessments, in addition to a gradebook that monitors student success and time on task. Having 
access to comprehensive online educational resources and a course management system specific to 
mathematics has been instrumental in convincing hesitant mathematics educators to try teaching in the 
online environment. 

Unfortunately, low retention rates in online freshman-level mathematics courses are a concern, 
especially at postsecondary institutions that serve academically unprepared students (Xu & Jaggars, 2011; 
Zavarella & Ignash, 2009). In a study of community college students enrolled in developmental 
mathematics courses, Zavarella and Ignash (2009) observed that students enrolled in computer-based 
courses were more likely to withdraw than students enrolled in lecture-based courses. Similarly, Xu and 
Jaggars (2011) found that community college students enrolled in online freshman-level mathematics 
courses experienced higher attrition rates and were less likely to earn grades of C or better than those 
students enrolled in face-to-face offerings. Both studies indicated that online instruction, as it was 
structured in these cases, may not be as effective as face-to-face instruction for students who are 
unprepared for freshman-level mathematics courses. 

Regardless of the issues with reduced retention, students in higher education continue to enroll in 
online courses. According to the latest report on the status of online education from the Babson Survey 
Research Group, approximately 33.5% of all college students enroll in at least one online class (Allen & 
Seaman, 2014). Furthermore, while the annual growth rate of overall enrollment in higher education from 
fall 2011 to fall 2012 was 1.2%, the annual growth rate of online enrollment in higher education over the 
same time period was 6.1%. Given that online education continues to play an important role in 
institutions of higher education, it is imperative that mathematics faculty determine whether or not 
instruction provided in online freshman-level mathematics courses is meeting students’ needs. If it is 
determined that online mathematics instruction is ineffective, can mathematics faculty identify and 
incorporate course design techniques that encourage student engagement, persistence, and learning? 

Literature Review 

Mathematics anxiety and negative attitudes towards mathematics can have a negative impact on 
student success in university-level mathematics courses. Nunez-Pena and Suarez-Pellicioni (2013) 
observed students who failed a mathematically challenging university course had higher mathematics 
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anxiety and higher course anxiety than students who passed the university course. Furthermore, they 
noted students who failed the course had lower self-confidence in and motivation toward mathematics 
than students who passed. While such attitudinal and emotional factors can impact student engagement 
and persistence in mathematics, Fumer and Gonzalez-DeHass (2011) indicated creating a mastery- 
oriented classroom can help reduce these anxieties. Specifically, they recommended teachers establish a 
class climate that emphasizes thinking rather than rote memorization, encourages students to ask 
questions, and allows students to engage in activities that allow them to learn from and correct their 
mistakes. 

Mathematics educators interested in creating a class climate that not only reduces student 
anxieties, but also improves student learning outcomes, have experimented with text-based, interactive 
multimedia tools such as ALEKS and MyMathLab (Aichele, Francisco, Utley, & Wescoatt, 2011; 
Brewer, 2009; Klein, 2005; Taylor, 2008; Xu, Meyer, & Morgan, 2008). While both web-based tools 
emphasize content mastery and provide students with immediate feedback that offers suggestions for 
where student errors may have occurred in the solution process, best practice in the use of these 
technologies in mathematics courses has yet to be determined. Taylor (2008) concluded that college 
freshmen enrolled in developmental mathematics courses using ALEKS experienced a reduction in 
anxiety and an improvement in attitude toward mathematics. Furthermore, results indicated that some 
students’ mathematics achievement was also improved. Similarly, Xu et al. (2008) noted that students in 
a graduate-level educational statistics course using a blended learning approach and ALEKS perfonned 
better than students enrolled in a more traditional course. Flowever, Xu et al. also discovered that low- 
and medium-performing students were more likely to perceive a misalignment between the course 
textbook and the online multimedia tools. Some students admitted to intentionally “working the system” 
in order to reduce the amount of time spent engaging with the online tools. And, although students may 
have appreciated being able to continuously review the material in the online environment, they 
acknowledged the importance of observing the instructor as problem solutions were being presented, as 
well as the importance of having their own work evaluated by the instructor so that misconceptions could 
be more accurately identified and corrected. 

Similar issues have been noted in freshman-level mathematics courses that use MyMathLab 
(Aichele, Francisco, Utley, & Wescoatt, 2011; Brewer, 2009; Klein, 2005). Brewer (2009) found 
students who enrolled in face-to-face College Algebra courses that used MyMathLab for homework 
experienced frustration not only with the software’s inability to provide more infonnative feedback when 
answers were incorrect, but also the level of precision required by the software in order to earn full credit. 
Overall, the mathematics achievement of students using online homework was comparable to the 
mathematics achievement of students using textbook homework. However, students who used 
MyMathLab for homework and who were academically unprepared or repeating the course did 
outperform their counterparts who used textbook homework. In a second study that explored the use of 
MyMathLab in College Algebra, Klein (2005) concluded that supplementing traditional College Algebra 
instruction with the computer-assisted instruction embedded within MyMathLab did not result in a 
statistically significant improvement in student achievement, nor did it improve student attitude toward 
mathematics. A third study by Aichele et al. (2011) focused on students’ perceptions of the usefulness of 
the educational resources embedded in MyMathLab. Both Klein (2005) and Aichele et al. (2011) noted 
that students preferred the View an Example and Help Me Solve This features embedded within 
MyMathLab. Furthermore, according to Aichele et al. (2011), students indicated that they preferred these 
two features because they provided step-by-step procedures for the solution process, with the View an 
Example feature being the most preferred since it required less work on the part of the student. Few 
students indicated that these features enhanced their conceptual understanding of the reasoning behind the 
solution process. In addition, the majority of students surveyed considered the publisher-generated videos 
to be one of the least beneficial resources since watching them required more time on the part of the 
student. 
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Research indicates teaching presence in the online classroom, as defined by Anderson, Rourke, 
Garrison, and Archer (2001), plays an important role in the development of a class climate that provides 
students with a rewarding educational experience. In a study of online student satisfaction, all three 
categories of teaching presence—design and organization, facilitating discourse, and direct instruction— 
were found to be highly correlated with course satisfaction and student perception of learning (Shea, 
Pickett, & Pelz, 2003). Data collected by Hosier and Arend (2012) indicated that online students believed 
teaching presence influences their critical thinking. In yet another study, teaching presence was found to 
be a strong predictor of student affective learning, motivation, and perceived cognition in online 
undergraduate and graduate classes (Baker, 2010). 

As college and university faculty across disciplines have gained experience in teaching and 
learning in the online environment, intentional course design techniques and instructional activities that 
tend to promote teaching presence and student motivation have surfaced. Lehman and Conceic^ao (2014) 
noted that one such course design technique involves dividing the course content into pieces called 
“chunks” and carefully arranging these chunks into modules having a similar structure. Designing the 
online course in this manner reduces cognitive overload and allows students to focus on the content being 
presented. Other intentional design techniques noted in online teaching literature include anticipating 
student needs and addressing them in orientation activities, providing clear expectations and well- 
established deadlines early in the course, and providing prompt and personalized feedback (Fish & 
Wickersham, 2009; Jaggars, Edgecombe, & Stacey, 2013; Lehman & Concei^ao, 2014). 

While online instructors across disciplines agree that the implementation of intentional course 
design techniques can enhance teaching presence and improve student engagement and persistence, the 
specific design techniques and instructional activities that best support the online learning environment 
may differ by discipline. In an effort to identify specific instructor behaviors used to promote teaching 
presence in online science courses, Stone and Chapman (2006) engaged in formal interviews with three 
undergraduate science instructors, each having at least five years of online teaching experience. Based on 
the interview data, Stone and Chapman were able to define instructor presence within three constructs: 
course content, instructor’s role, and student needs. The instructors who were interviewed believed that 
they were able to create and sustain teaching presence in online science courses by placing themselves in 
the role of content provider (i.e., the individual who has ownership of and control over the course content, 
and the ability to make content changes and updates), as well as the role of facilitator of learning; 
incoiporating their own personal educational materials, as well as a variety of multimedia tools; 
maintaining instructor-student interaction by supplying prompt and frequent feedback; providing an 
effective student support structure; and organizing content and structuring the course to allow for self- 
directed learning. 

Just as instructors differ in their perceptions of what design techniques and instructional activities 
are most appropriate for teaching and learning their discipline in an online environment, students enrolled 
in online courses also differ in their perceptions of what instructional activities best support their learning 
needs. In an analysis of student behaviors in online general studies courses from various disciplines, 
Finnegan, Morris, and Lee (2009) discovered that while students must be willing to spend time engaging 
in online activities if they are to be successful, the online activities they use to engage in the learning 
process may differ by discipline. Students who successfully completed online English, communication, 
and social science courses spent more of their time participating in online discussions (e.g., posting their 
own questions, reading and replying to other students’ posts), while students who successfully completed 
online science and mathematics courses spent more of their time viewing content pages and posting their 
own questions. Simonds and Brock (2014), in an exploration of student learning preferences in online 
graduate education courses, concluded that while younger students preferred interactive online learning 
activities, older students preferred to learn from recorded video lectures. Given the positive impact of 
providing recorded video lectures for asynchronous viewing in face-to-face undergraduate courses that 
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have significant mathematical content (Brecht, 2012; Cascaval, Fogler, Abrams, & Durham, 2008), it may 
be that students enrolled in online mathematics and science courses would also benefit from the inclusion 
of video lectures as instructional activities. 

Although mathematics educators have experimented with the use of text-based multimedia tools 
such as ALEKS and MyMathLab in the teaching and learning of freshman-level mathematics, additional 
research is needed to determine the most effective way to use these tools in the online general studies 
mathematics course. Since text-based tools such as MyMathLab provide students access to 
comprehensive educational resources, can these publisher-generated resources, by themselves, adequately 
strengthen students’ conceptual understanding to ensure student success? Does the availability of the 
publisher-generated educational resources diminish the visibility of the course instructor’s teaching 
presence, and if so, what can course instructors do to re-establish their teaching presence? If students 
have a tendency to take shortcuts to reduce time on task, is it best to restrict the publisher-generated 
educational resources that are available to them within the online homework system? If certain publisher¬ 
generated educational resources are disabled within the online homework system, what instructor¬ 
generated educational resources should be provided in their place to facilitate student learning? The 
purpose of this study was to determine if an online College Algebra course relying heavily on publisher¬ 
generated educational resources can be redesigned to make the course instructor’s teaching presence more 
visible and hence, better meet the needs of unprepared students. Consequently, the study focused on the 
following research questions: 

1. Can student performance in an online College Algebra course that relies heavily on text- 
based multimedia tools be improved by replacing the publisher-generated educational 
resources with instructor-generated video lectures? 

2. Can student attrition in an online College Algebra course that relies heavily on text-based 
multimedia tools be reduced by replacing the publisher-generated educational resources 
with instructor-generated video lectures? 

3. Can teaching presence in the fonn of instructor-generated video lectures serve as a 
predictor of student success in online College Algebra? 

Method 

Participants 

Study participants included students who self-enrolled in the online College Algebra courses 
offered at an open-enrollment university in the Midwestern United States over a two-year period. All 
courses were assigned to the author and were available for enrollment throughout the regular registration 
periods. The chairperson of the university’s Institutional Research Board infonned the author that 
standard educational practices were exempt from the IRB approval requirement. Therefore, since the 
author was simply teaching the online course differently than how it had been taught previously, IRB 
approval was not required. This study is limited by the fact that students were not randomly assigned to 
the courses included in the analysis, but perhaps strengthened by the fact that the author has been the only 
faculty member assigned to the online offerings of College Algebra at this university. 

Sixty-nine students enrolled in the original online courses offered during the first year (i.e., 
Spring 2012 and Fall 2012). Eighty-seven students enrolled in the redesigned online courses offered 
during the second year (i.e., Fall 2013 and Spring 2014). Although students are not required to have 
documented ACT scores on file to enroll in the university, the current course prerequisite for College 
Algebra at the university is an ACT Math sub-score of at least 20, or the equivalent (i.e., a grade of C or 
better in the university’s developmental mathematics courses). Given the strong positive relationship 
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between ACT scores and college student success (Bleyaert, 2010; Richardson, Abraham, & Bond, 2012), 
students having no documented ACT scores on file with the university were deleted from the samples. 
Furthermore, students who remained on the course roster but did not actually complete the course (i.e., 
they stopped attending the course but did not officially withdraw) were also deleted. Table 1 provides an 
overview of the student profiles included in all statistical analyses. 

Table 1 . Online College Algebra Student Profiles, by Course Design 



Original 

(Spring 2012, Fall 2012) 

Redesigned 

(Fall 2013, Spring 2014) 

Number of students 

51 

44 

Mean ACT Math 

17.76 (50=2.43) 

18.52 (50=3.19) 

Mean Age 

23.47 (50=6.74) 

26.09 (50=9.02) 

Gender 

Female 67% 

Female 68% 


Male 33% 

Male 32% 


Freshman 35% 

Freshman 32% 

Student Rank 

Sophomore 26% 

Sophomore 27% 


Junior 26% 

Junior 18% 


Senior 14% 

Senior 23% 

Off-Campus 

86% 

86% 

Repeating Course 

35% 

34% 


While the mean ACT Math sub-score for the redesigned course was higher than the mean ACT 
Math sub-score for the original course, the difference was not statistically significant (H A : pj ^ p 2 ; t(19)=- 
1.29; p= 0.202). A similar difference was noted in the overall campus mean ACT Math sub-scores during 
the two time periods (i.e., 19.81 and 20.07, respectively). A comparison of the campus mean ACT Math 
sub-scores to the mean ACT Math sub-scores in the online offerings of College Algebra indicates that the 
online offerings of College Algebra at this open-enrollment university tend to attract mathematically 
weaker students. 

The mean age of students enrolled in the redesigned course was higher than the mean age of 
students enrolled in the original course, but the difference was not statistically significant (H A \ g, ^ p 2 ; 
t(78)=-1.58; p=0.117). Comparisons of student differences in gender ('yj (1, A=95)=0.025; £>=0.875), 
student rank (% 2 (3, A ,7 =95)= 1.754; £>=0.625), location of residence, and whether or not the College 
Algebra course was being repeated (~/_ 2 (1, ;V=95)=0.015; £>=0.902), also indicated no statistically 
significant differences. Furthermore, although approximately 35% of the students enrolled during the 
four-semester period were repeating College Algebra, none of the students were repeating the online 
offering of the course. 

While this study is limited by the fact that students were not randomly assigned to the courses 
included in the analysis, the evidence suggests the two samples were independent and drawn from the 
same population, and thus, may be considered equivalent for the purposes of this study. 

Technology 

All online courses over the two-year period were required to have access to MyMathLab, an 
online multimedia educational system designed by Pearson Education to accompany its published 
mathematics textbooks. Students enrolled in the redesigned online College Algebra course were also 
required to use Moodle, the university’s online learning management system. All instructor-generated 
video lectures in the redesigned course were created by projecting PowerPoint presentations onto a 
SMART Podium Annotation Monitor and recording all handwritten solutions and oral explanations with 
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Panopto, a lecture capture system. Instructor-generated explanations for how to use the graphing 
calculator were also recorded using Panopto and a document camera. 

Course Design 

Students enrolled in both the original online College Algebra course and the redesigned online 
College Algebra course were required to have access to the multimedia educational system that 
accompanies the course textbook adopted by the university. This text-based multimedia educational 
system provided students with access to the e-book (i.e., an electronic copy of the textbook) and an online 
homework system. 

Original Online College Algebra Course, Spring 2012 and Fall 2012. In the original online 
College Algebra course, the primary sources for course content and instruction were the publisher¬ 
generated learning aids enabled within the online homework exercises provided by the text-based 
multimedia educational system. Each exercise within the online homework system offered students a 
variety of publisher-generated learning aids to assist them in obtaining the correct answer for that 
particular exercise. Publisher-generated learning aids available for each homework exercise included a 
15-30 minute video lecture provided by one of five different lecturers, an animation, a completed 
example, a guided tutorial, a link to the e-book, and a mechanism for asking the instructor a question via 
email. Both the video lecture and the animation provided a general overview of the topic being assessed 
by the given homework exercise, while the completed example provided a step-by-step solution for a 
problem identical in type, difficulty level, and conceptual scope. The guided tutorial assisted students in 
working the given homework exercise, requiring that the student then complete an algorithmically- 
generated identical problem on their own to earn credit for that exercise. 

The course instructor’s teaching presence in the original online College Algebra course consisted 
of supplemental instructor-generated educational resources (i.e., typed lecture notes, offline exam review 
materials, and video recordings of handwritten solutions to problems included in the offline exam review 
materials) uploaded into the text-based multimedia educational system and available on the first day of 
the semester, instructor-student interaction in the form of frequent instructor-initiated group email 
messages (at least one per week) and periodic individualized course status email messages, and instructor- 
student interaction in the form of prompt email responses to student questions about content (generally 
within 12 hours). The importance of email as the main mode of communication was emphasized 
frequently at the start of each semester. Except for the video recordings of handwritten solutions to 
problems included in the offline exam review materials, instructor-student interaction was limited to 
typed communication in the form of typed lecture notes or email correspondence. 

In the original online College Algebra course, students were not required to engage with any 
specific educational resource. They were given the freedom to decide whether or not to engage with a 
publisher-generated learning aid associated with an online homework exercise or an instructor-generated 
resource. 

Redesigned Online College Algebra Course, Fall 2013 and Spring 2014. In the redesigned 
online College Algebra course, the primary sources for course content and instruction were instructor¬ 
generated video lectures and coordinated note-taking sheets that were organized within the university- 
adopted course management system (i.e., outside of the publisher’s multimedia educational system). The 
course content was assembled into five units, each unit consisting of five to six similarly-structured 
modules, a course design technique that has been shown to promote student motivation (Lehman & 
Conceipao, 2014). Each unit of modules became visible to the students only when it was time to begin 
working on that unit. The offline exam review materials provided by the instructor in the original online 
College Algebra course were also provided in the redesigned College Algebra course and were attached 
to the end of each unit. 
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Each module consisted of a set of instructor-generated guided note-taking sheets and at most five 
instructor-generated video lectures, each 5-40 minutes in length, with 65% of the videos being less than 
20 minutes and 91% of the videos being less than 30 minutes. The content of the note-taking sheets was 
carefully coordinated with the content of the instructor-generated video lectures. Students were required 
to print off the note-taking sheets and complete them while watching the corresponding PowerPoint 
presentations that provided complete instruction for the algebraic concepts being addressed in that module 
and solutions to several relevant examples, emphasizing throughout the importance of understanding why 
each step in a solution is being performed. All 91 video lectures (i.e., over 26 hours of recorded lecture) 
were created in an empty room using an annotation monitor, a document camera, and a lecture capture 
system. As the instructor displayed the PowerPoint slides, concepts were discussed and examples were 
completed directly on the annotation monitor, comparable to the instruction provided in the face-to-face 
environment. All handwritten solutions and verbal explanations were recorded by the lecture capture 
software, allowing students the opportunity to pause, rewind, fast forward, and replay as needed. The 
note-taking sheets were graded on completion four times during the semester and returned to the student. 

A link to the text-based multimedia educational system was included within the university- 
adopted course management system. While the original online College Algebra course enabled all of the 
publisher-generated learning aids within the online homework exercises provided by the text-based 
multimedia educational system, the redesigned course treated the publisher-generated learning aids as 
supplemental resources that accompany the course textbook. That is, the only publisher-generated 
learning aids enabled within the online homework in the redesigned course were the link to the e-book 
and the mechanism for asking the instructor a question via email. Although the publisher-generated video 
lectures, completed examples, and guided tutorials were not enabled within the homework, these learning 
aids were accessible via the online version of the textbook, a fact that was not publicized by the course 
instructor. The online version of the textbook was different from the e-book in that it included embedded 
li nk s to not only relevant publisher-generated video lectures, but also interactive examples that provide 
access to other publisher-generated learning aids (i.e., completed examples and guided tutorials). While 
the e-book could be accessed within the homework system, the online version of the textbook with the 
embedded learning aids was only accessible when the homework system was closed. 

At the beginning of each offering of the redesigned online College Algebra course, the instructor 
emphasized the importance of referring to the instructor-generated videos and their corresponding note¬ 
taking sheets when questions emerged while completing homework assignments. Students needing 
additional assistance beyond what the videos and note-sheets could provide were encouraged to email the 
instructor a detailed question. The instructor also encouraged students in the redesigned course to use 
their smartphones to send pictures of their work to the instructor rather than typing their questions, as was 
the norm in the original course. This approach made it easier for students to clearly communicate 
mathematical symbolism and easier for the instructor to locate student errors. 

The course instructor’s teaching presence in the redesigned online College Algebra course 
consisted of instructor-generated video lectures and their respective note-taking sheets (as well as the 
offline exam review materials) organized within the university-adopted course management system, 
instructor-student interaction in the form of frequent instructor-initiated group email messages (at least 
one per week) and periodic individualized course status email messages, and instructor-student interaction 
in the form of prompt email responses to student-initiated questions about content (generally within 12 
hours). Again, the importance of email as the main mode of communication was emphasized frequently 
at the start of each semester. 

In the redesigned online College Algebra course, students were required to complete guided note¬ 
taking sheets while watching over 26 hours of instructor-generated video lectures. Although students 
were given the freedom to decide whether or not to engage with a publisher-generated learning aid, these 
learning aids were only accessible outside of the homework system. 
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Table 2 summarizes how the original online College Algebra course compares to the redesigned 
online College Algebra course in terms of how the course instructor’s teaching presence was facilitated 
according to the eight descriptors identified by Stone and Chapman (2006). 

Table 2. Efforts to Facilitate Course Instructor’s Teaching Presence, by Course Design 


1. Via the instructor’s role as 
content provider and subject 
matter expert 

2. By using materials unique to 
the instructor 


3. Through the creation of one’s 
own learning objects 

4. By designing an effective 
student support structure 

5. Through the instructor’s role as 
facilitator of learning 


6. By implementing high 
instructor immediacy behaviors in 
feedback 

7. Through the use of a variety of 
media formats 


Original 


Type-set lecture 

(201 pages) (Optional) 

notes 

Type-set lecture 

(201 pages) (Optional) 

notes 

Type-set lecture 

(201 pages) (Optional) 

notes 

Offline and online 
materials 

review 

Instructor-initiated weekly 

group email announcements; 
Instructor responses to student- 
initiated questions 

Email responses to student 
questions (generally within 12 
hours) 

Publisher-generated 
aids enabled within 
homework 

learning 

online 


Redesigned _ 

Instructor-generated video 
lectures (over 26 hours) 

(Required) 

Instructor-generated video 
lectures (over 26 hours) 

(Required) 

Guided note-taking sheets (182 
pages) graded on completion 
(Required) 

Offline and online review 

materials 

Instructor-initiated weekly 
group email announcements; 
Instructor responses to student- 
initiated questions 

Email responses to student 

questions (generally within 12 
hours) 

Publisher-generated learning 
aids accessible within online 
textbook but disabled within 
online homework 


8. By organizing content and 
structuring the course for self- 
directed learning 


Course materials organized 
within publisher-provided 
multimedia system ; 

Homework assignments and 
course materials visible from 
first day 


Course materials organized 
outside of publisher-provided 
multimedia system'. 

Homework assignments and 
course materials become visible 
when unit begins 


Adapted from Stone et al. (2006) 


As noted in Table 2, the aspects of teaching presence related to facilitating discourse were identical 
between the two design models. However, the two design models differed in how they facilitated the 
course instructor’s teaching presence in relation to design and organization, and direct instruction. 


Course Grading Policies and Assessment Components 

Course grades in the original online College Algebra course were computed as follows: Online 
Homework (10%), Online Quizzes (10%), Online Exams (30%), Handwritten Midterm Exam (25%), and 
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Handwritten Common Comprehensive Final Exam (25%). In the redesigned online College Algebra 
course, the Online Homework was reduced to 5% of the overall course grade, with Class Participation 
(i.e., the note-taking sheets graded on completion) accounting for the remaining 5%. The weights of all 
other assessment components were the same. 

The online assessment components in both the original and the redesigned courses were 
intentionally structured to create a class climate that emphasizes a mastery-oriented approach to learning. 
Prior to the due date for a given online homework assignment, students had unlimited attempts to 
correctly answer each question. During each two-day online quiz window, students were allowed to take 
the quiz up to three times. During each two-day online exam window, students were allowed to take the 
exam only once. 

The problems selected for inclusion on all online assessments (i.e., homework, quizzes, and 
exams) were identical in both the original and the redesigned courses. The online quizzes and online 
exams were non-proctored assessments, while the handwritten midterm exam and handwritten final exam 
were proctored by either the instructor or by an institution listed as a participant of the Consortium of 
College Testing Centers. 

Beginning in the Spring 2012 semester, the university mathematics department implemented a 
handwritten common final exam to be given in all College Algebra offerings. The common final exam 
was given in both the original and the redesigned courses. Associated with this common final exam is a 
policy that requires students to earn at least 50% on the exam in order to earn a passing grade in the 
course. The final exam consists of 20 typical College Algebra problems. Each problem is randomly 
selected from a pool of 2-3 algorithmically-generated problems that are similar in difficulty level and 
conceptual scope. In an effort to better prepare the online students for the format of the high-stakes final 
exam, the instructor created the handwritten midterm exam to include problems comparable to those that 
might appear on the common final exam. The midterm exam was also given in both the original and the 
redesigned courses. The handwritten midterm exam for Spring 2012 served as the model for the midterm 
exams given during the Fall 2012, Fall 2013, and Spring 2014. That is, questions selected for inclusion 
on the midterm exams after the initial Spring 2012 semester were similar in difficulty level and 
conceptual scope to those questions on the original midterm exam. To maintain consistency in the 
grading of the handwritten exams given in both the original and redesigned courses, detailed grading 
rubrics were followed after removing all student names. 

Time on Task 

Given the tendency for successful online students to spend more time engaging with online 
learning activities (Finnegan, Morris, & Lee, 2009), a measurement for time on task was determined for 
all enrolled students. 

Original Online College Algebra Course, Spring 2012 and Fall 2012. Time on task (in hours) 
for each student in the original online College Algebra course was measured by the time tracking feature 
of the text-based multimedia educational system that monitors student activity within online assessments 
(i.e., online homework, online quizzes, and online exams). Consequently, when a student chose to engage 
with a publisher-generated learning aid (i.e., a publisher-generated video, an animation, a completed 
example, a guided tutorial, or the e-book) while working on online homework, the time spent interacting 
with the learning aid was included in the measurement of time on task. Time spent engaging with a 
publisher-generated learning aid outside of the online homework assignment, time spent reviewing past 
online assessments, and time spent working offline were not included in this measurement. 

Redesigned Online College Algebra Course, Fall 2013 and Spring 2014. Time on task (in 
hours) for each student in the redesigned online College Algebra course consisted of the sum of two 


Online Learning Volume 19 Issue 3 (2015) 


79 


quantities—an approximation for the time spent watching the instructor-generated video lectures and the 
time on task as measured by the time tracking feature of the text-based multimedia educational system. 
When the video lectures were inadvertently made public, the lecture capture system did not retain user 
statistics, and hence, an approximation for the time spent watching the instructor-generated video lectures 
was computed. Since the class participation grade for the redesigned course was determined by the 
percentage of the note-taking sheets completed, and the note-taking sheets could not be completed 
without watching the video lectures, the time spent watching the instructor-generated video lectures was 
approximated by multiplying the total number of hours in the video collection (i.e., 26 hours) by the class 
participation grade. 

As with the original online College Algebra course, the time tracking feature of the text-based 
multimedia educational system monitored student activity within online assessments (i.e., online 
homework, online quizzes, and online exams). When a student chose to engage with the e-book (i.e., the 
only publisher-generated learning aid enabled within the online homework in the redesigned College 
Algebra course) while working on online homework, the time spent interacting with the e-book was 
included in the measurement of time on task. Time spent engaging with a publisher-generated learning 
aid outside of the online homework assignment, time spent reviewing past online assessments, and time 
spent working offline were not included in this measurement. 

Data Analysis 

Student profde data suggests that the two samples were independent and drawn from the same 
population, and thus, may be considered equivalent for the purposes of this study. The level of 
significance used in all data analysis was a=0.05. 

Student Performance. To address the question of whether student performance in an online 
College Algebra course relying heavily on text-based multimedia tools can be improved by replacing the 
publisher-generated educational resources with instructor-generated video lectures, five different one- 
tailed t-tests for independent samples were used to compare the original online College Algebra course to 
the redesigned College Algebra course on all assessment components common to both courses: online 
homework, online quizzes, online exams, handwritten midterm exam, and handwritten final exam. 

Online Homework. The student overall score on online homework was determined by 
computing the percentage correct on 25 equally-weighted homework assignments. 

Online Quizzes. The student overall score on online quizzes was determined by computing the 
percentage correct on five equally-weighted quizzes. 

Online Exams. The student overall score on online exams was determined by computing the 
percentage correct on three equally-weighted exams. 

Handwritten Midterm Exam and Handwritten Final Exam. Student scores on both handwritten 
exams were determined by awarding points according to detailed grading rubrics, summing all points 
earned, and computing the corresponding percentage. 

Time on Task. A /-test for independent samples was used to compare the time on task (in hours) 
for students in the original online College Algebra course with the time on task for students in the 
redesigned College Algebra course. 

Course Grade Distributions and Student Attrition. Given that many university degree 
programs require a grade of C or better in general studies mathematics, letter grades of A, B, and C were 
considered passing grades in this analysis. A chi-square test was used to compare the overall course pass 
rate of the original online College Algebra course to the overall course pass rate of the redesigned College 
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Algebra course. Similarly, a second chi-square test was used to compare overall course pass rates when 
all 156 enrolled students were taken into consideration. 

The student attrition rate was defined to be the percentage of students officially withdrawing from 
the course. To address the question of whether student attrition in an online College Algebra course 
relying heavily on text-based multimedia tools can be reduced by replacing the publisher-generated 
educational resources with instructor-generated video lectures, a chi-square test was used to compare the 
student attrition rate of the original online College Algebra course to the student attrition rate of the 
redesigned College Algebra course. Similarly, a second chi-square test was used to compare student 
attrition rates when all 156 enrolled students were taken into consideration. 

Teaching Presence as a Predictor of Student Success. To address the question of whether 
teaching presence in the form of instructor-generated video lectures can serve as a predictor of student 
success in online College Algebra, a multiple regression analysis was performed. The independent 
variables for the regression model included design (original=0, redesigned=l), time on task (in hours), 
and ACT Math sub-scores (1-36). The dependent variable was student learning as measured by the 
overall course percentage computed according to the course grading policies outline in the course syllabi. 

Results 

Assessment Components 

Online Homework. Results indicated student overall scores on online homework in the original 
online College Algebra course (47=83.43, 50=16.95) were not statistically significantly different from 
student overall scores on online homework in the redesigned online College Algebra course (47= 82.13, 
50=17.12; t(54)=0.29,£>=0.613, one-tailed). 

Online Quizzes. Results indicated student overall scores on online quizzes in the original online 
College Algebra course (47=67.41, 5D=23.55) were statistically significantly lower than student overall 
scores on online quizzes in the redesigned online College Algebra course (47=76.63, 50=12.95; t(45)=- 
1.86, p=0. 035, one-tailed). The magnitude of the difference in means was small (mean difference=-9.22, 
95% upper bound: -0.87; r 2 =0.06). 

Online Exams. Again, results indicated student overall scores on online exams in the original 
online College Algebra course (47=64.47, 50=21.44) were statistically significantly lower than student 
overall scores on online exams in the redesigned online College Algebra course (47=78.25, 50=12.40; 
t(47)=-3.01, p=0.002, one-tailed). The magnitude of the difference in means was moderate (mean 
difference=-13.78, 95% upper bound: -6.09; r 2 =0.16). That is, course design explains 16% of the 
variation in student overall scores on online exams. 

Handwritten Midterm Exam. Student scores on the handwritten midterm exam in the original 
online College Algebra course (,17=50.00, 50=21.84) were statistically significantly lower than student 
scores on the handwritten midterm exam in the redesigned online College Algebra course (47=65.75, 
50=15.79; t(52)=-3.14, £>=0.001, one-tailed). The magnitude of the difference in means was moderate 
(mean difference—15.75, 95% upper bound: -7.36; r 2 =0.16). That is, course design explains 16% of the 
variation in student scores on the handwritten midterm exam. 

Handwritten Final Exam. Student scores on the handwritten final exam in the original online 
College Algebra course (47=44.93, 50=26.36) were statistically significantly lower than student scores on 
the handwritten final exam in the redesigned online College Algebra course (47=59.94, 50=17.29; 7(50)=- 
2.56, £>=0.007, one-tailed). The magnitude of the difference in means was moderate (mean difference=- 
15.01, 95% upper bound: -5.20; r 2 =0.12). That is, course design explains 12% of the variation in student 
scores on the handwritten final exam. 
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Time on Task 


Results indicated that while the mean number of hours spent on task by students in the original 
online College Algebra course (M=60.80, S0=52.29) was lower than the mean number of hours spent on 
task by students in the redesigned online College Algebra course (M=73.78, 577=32.99), the difference 
was not statistically significant (/(49)=-l.13,/?=0.132, one-tailed). 

Table 3 summarizes the results obtained in the comparison of the original online College Algebra 
course and the redesigned online College Algebra course on all assessment components and time on task. 


Table 3. Computed Means for Assessment Components and Time on Task, by Course Design 



Original 
(N=5 1) 

Redesigned 
(N= 44) 

P 

r 2 

Online Homework 

83.43% 

82.13% 

0.613 

na 

Online Quizzes 

67.41% 

76.63% 

0.035 

0.06 

Online Exams 

64.47% 

78.25% 

0.002 

0.16 

Handwritten Midterm 

50.00% 

65.75% 

0.001 

0.16 

Handwritten Final 

44.93% 

59.94% 

0.007 

0.12 

Time on Task 

60.80 hrs 

73.78 hrs 

0.132 

na 

Course Grade Distributions and Student Attrition 




Overall course grade distributions, sorted by course design, are provided in Table 4. 


Table 4. Grade Distributions, by Course Design 





Original 

Redesigned 




(7V=51) 

(N=44) 

Difference 

A 

5.88% 

4.55% 

-1.34 


B 

5.88% 

6.82% 

0.94 


C 

3.92% 

27.27% 

23.35 


Grades of A, B, and C together 15.69% 

38.64% 

22.95 


D 

15.69% 

11.36% 

-432 


F 

27.45% 

11.36% 

-16.09 


W 

41.18% 

38.64% 

-2.54 



The course pass rate for the redesigned online College Algebra course was statistically significantly 
higher than the course pass rate for the original online College Algebra course (yj (1, /V=95)=6.416; 
j?=0.011). Furthermore, while the course pass rate for the original online College Algebra course was 
22.45 percentage points below the campus-wide College Algebra pass rate during the same time period, 
the course pass rate for the redesigned online College Algebra course was only 4.39 percentage points 
below the campus-wide College Algebra pass rate during the same time period. The student attrition rate 
for the redesigned online College Algebra course, however, was not statistically significantly lower than 
the student attrition rate for the original online College Algebra course (% 2 (1, /V=95)=0.064; /?=().801). 
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The grade distributions for all 156 students enrolled during the two-year period were also 
computed. The course pass rate for the redesigned online College Algebra course (25.29%, N= 87) was 
almost double the course pass rate for the original online College Algebra course (13.04%, N= 69), though 
the difference was not statistically significant (% 2 (1, ,V=156)=3.623; /?=0.057). And, while the attrition 
rate for the redesigned online College Algebra course (27.59%, N= 87) was not statistically significantly 
lower than the attrition rate for the original online College Algebra course (42.03%, N= 69; X 0, 
A=156)=3.578; p=0.059), the attrition rate did drop 14.44 percentage points. 

Teaching Presence as a Predictor of Student Success 

Results from a multiple regression analysis indicated that the linear combination of design, time 
on task, and ACT Math sub-score was statistically significantly related to the overall course percentage 
(F( 3, 53)=8.47, p < 0.001). The adjusted correlation coefficient was 0.29 indicating that 29% of the 
variability in student learning was accounted for by design, time on task, and ACT Math sub-score. All 
three independent variables were statistically significant predictors of student learning (p < 0.05) as seen 
in Table 5. 


Table 5. Summary of Multiple Regression Results 


Model 

Unstandardized 

Coefficient 

Standardized 

Coefficient 

t 

P 

Intercept 

20.64 

12.97 

1.59 

0.118 

Design 

9.401 

3.838 

2.45 

0.018 

Time on Task 

0.11868 

0.004271 

2.78 

0.008 

ACT Math Score 

1.6643 

0.6784 

2.45 

0.017 


Discussion 

This study examined the impact of instructor-generated video lectures on student performance 
and student attrition in an online College Algebra course at an open-enrollment university. The original 
online College Algebra course placed the publisher-generated educational resources in the role of content 
provider by enabling all publisher-generated learning aids within the online homework system and 
treating instructor-generated educational materials as supplemental resources. In contrast, the redesigned 
online College Algebra course enhanced the course instructor’s teaching presence by requiring students to 
complete instructor-generated guided note-taking sheets while watching over 26 hours of instructor¬ 
generated video lectures, treating publisher-generated learning aids as supplemental resources by 
removing them from within the online homework system. 

In studies by Klein (2005) and Aichele, Francisco, Utley, and Wescoatt (2011), students having 
access to the publisher-generated learning aids embedded within the MyMathLab homework system 
found the View an Example and Help Me Solve This features to be the most valuable because they provide 
step-by-step solutions to very similar problems. Aichele et al. (2011) suggested that when students 
associate procedural learning with success, the belief that mathematics is nothing more than the rote 
memorization of steps in a process is reinforced. In removing the student-preferred, publisher-generated 
learning aids from within the online homework system and replacing them with instructor-generated 
video lectures that provided complete instruction for the algebraic concepts being addressed, procedural 
learning was deemphasized and the importance of conceptual understanding in the learning of 
mathematics was highlighted. 
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Replacing publisher-generated learning aids with instructor-generated video lectures not only 
placed a greater emphasis on conceptual understanding, but also positioned the course instructor 
prominently in the role of content provider, enhancing the course instructor’s teaching presence in the 
online environment. The importance of course design and direct instruction in the online environment 
was noted by several earlier studies that indicated teaching presence is highly correlated with student 
perception of learning and critical thinking (Baker, 2010; Hosier & Arend, 2012; Shea et al., 2003). The 
results of the study described herein offer additional support to the contention that the course instructor’s 
teaching presence in the online environment and student learning are related. Specifically, while student 
performance on online homework in the original online College Algebra course was comparable to the 
student perfonnance on online homework in the redesigned online College Algebra course, students 
enrolled in the redesigned online College Algebra course performed statistically significantly better on all 
online and handwritten summative assessments (i.e., quizzes and exams) than students enrolled in the 
original online College Algebra course. 

Given that the students enrolled in the redesigned online College Algebra course perfonned better 
on all summative assessments, it is no surprise that the students enrolled in the redesigned course earned a 
statistically significant higher percentage of passing grades than the students enrolled in the original 
online College Algebra course. Furthermore, when the overall course grades of all 156 students enrolled 
during the two-year period were included in the grade distributions, it was noted that the student attrition 
rate in the course that incorporated the instructor-generated video lectures dropped considerably, an 
outcome that was also noted by Brecht (2012) in his study of the use of online video lectures as a 
supplement to a mathematics intensive face-to-face course. 

While the results suggest that the amount of time students in the redesigned online College 
Algebra course spent engaging with the online course content was not statistically significantly greater 
than the amount of time spent by students in the original online College Algebra course, this difference 
may be underestimated by the approximation used in measuring the total amount of time on task for the 
redesigned course. While there is no reason to believe the data collected by the text-based course 
management system inaccurately measured the amount of time spent working on online assessments, the 
actual time spent engaging with the video lectures is likely greater than the approximated value since it is 
known that students frequently pause and rewind the recorded lectures during the process of completing 
the guided note-taking sheets. And, although the measurement for time on task was computed 
consistently for each study participant according to the outlined method for their course, the accuracy of 
time on task, in general, is questionable since the measured value may include logged time during which 
the student was not actually engaging with the materials. Furthermore, the time on task measurement 
does not include time spent engaging with publisher-generated learning aids outside of the online 
homework assignments, time spent reviewing past online assessments, and time spent working offline. 

The results of this study also indicate that the design of the online College Algebra course, the 
amount of time a student spends engaging with the online course content, and the student’s ACT Math 
sub-score, together form a good linear model for predicting student learning as measured by the overall 
course percentage. That is, to increase the likelihood that a student will successfully pass online College 
Algebra, not only must students enter the course with an established amount of prerequisite knowledge 
and a willingness to commit a significant amount of time to learning the course material, but the instructor 
must also incorporate design techniques and instructional activities that create and sustain a strong 
teaching presence in the course. 

To illustrate the potential importance of the described redesign in online College Algebra, 
consider the following example. Suppose a student enrolled in the original online College Algebra course 
that centered around the publisher’s text-based multimedia tools had an ACT Math sub-score of 22 (i.e., 
the course prerequisite for College Algebra at the author’s university, effective fall 2015) and spent 61 
hours engaging with the online course content (i.e., the average amount of time on task for students 
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enrolled in the original online course). The regression model predicts this student’s overall course 
percentage to be 64.5%, resulting in a letter grade of D. As a comparison, suppose a student enrolled in 
the redesigned online College Algebra course that was organized around instructor-generated video 
lectures had an ACT Math sub-score of 19 and spent 74 hours engaging with the online course content 
(i.e., the average amount of time on task for students enrolled in the redesigned online course). The 
regression model predicts this student’s overall course percentage to be 70.4%, resulting in a letter grade 
of C. This suggests that the described redesign for online College Algebra may assist mathematically 
weaker students in learning the course content well enough to pass the course. 

Limitations 

Since the participants in this study enrolled themselves in the online sections of College Algebra 
offered by a public university and students cannot be forced to take an online course, there is the 
possibility of self-selection bias. Given the lack of random assignment to the different course design 
models, it is possible unmeasured factors influenced the outcomes of this study. Another limitation of 
this study is that many changes were made to the redesigned online College Algebra course, and hence, 
interactions between these changes may have occurred. For example, the redesigned online College 
Algebra course not only incorporated instructor-generated video lectures and the corresponding note¬ 
taking sheets, but also removed the publisher-generated learning aids from within the online homework 
system. 


To overcome these issues, a future research design coidd be implemented in which the self- 
enrolled students in a single high-enrollment online College Algebra course are randomly assigned to two 
distinct online courses. While both courses could incorporate the same instructor-generated video 
lectures and corresponding note-taking sheets, one course coidd disable the publisher-generated learning 
aids within the online homework system while the other course could enable the publisher-generated 
learning aids within the online homework system. 

Also, since the online course instructor was also the researcher, unintentional bias may have also 
influenced the outcomes. The best way to overcome this concern is to repeat the implementation of the 
redesigned model and observe the results over several more semesters. 

Conclusions 

The online College Algebra course offered by the open-enrollment university at the center of this 
study has a tendency to attract students who are mathematically weaker than the campus average. While 
these students might experience more success in a College Algebra course in a face-to-face format, for 
whatever reason they are choosing to enroll in the online course. As with any course—face-to-face or 
online—it is the responsibility of the instructor to incorporate course design techniques and instructional 
activities that are likely to encourage student engagement, persistence, and learning. This study suggests 
instructor-generated video lectures and coordinated note-taking sheets organized within modules may 
increase success among mathematically unprepared students enrolled in online freshman-level 
mathematics courses. 

Although text-based multimedia tools have much to offer students, instructors of online general 
studies mathematics courses that are structured around the multimedia tools accompanying course 
textbooks should proceed with caution. Care must be taken when designing an online course around 
these text-based multimedia tools to ensure that the instructor is placed prominently in the role of content 
provider. 
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